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ABSTRACT: 

A stochastic process called Vertex-Reinforced Random Walk (VRRW) is defined in Pe¬ 
mantle (1988a). We consider this process in the case where the underlying graph is an 
infinite chain (i.e., the one-dimensional integer lattice). We show that the range is almost 
surely finite, that at least 5 points are visited infinitely often almost surely, and that with 
positive probability the range contains exactly 5 points. There are always points visited 
infinitely often but at a set of times of zero density, and we show that the number of visits 
to such a point to time n may be asymptotically n a for a dense set of values aG (0,1). The 
power law analysis relies on analysis of a related urn model. 
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1 Outline of results 


For any process Xq, X\, X 2 , . .. taking values in the vertex set of a graph G (throughout 
this paper G = Z), we define the augmented occupation numbers 

n 

Z{n,v ) = 1 + 1 x i =v 
i =0 

to be the number of times plus one that the process visits site v up through time n. Let 
G be any locally finite graph, with the neighbor relation denoted by ~, and define vertex- 
reinforced random walk (VRRW) on G with starting point v E V{G ) to be the process 
{Xi : * > 0} such that Xq = v and 

P(X n+1 =x\X n ) = W Z{n r 

Hw~Xn Z ( n ’ W ) 

In other words, moves are restricted to the edges of G, with the probability of a move to a 
neighbor w being proportional to the augmented occupation of w at that time. 


This is a special case of the weighted VRRW, defined by Pemantle (1988a, 1992), where 
each oriented edge vw carries a nonnegative weight X(v,w), and the transition probabilities 
are given by 


P(V n+ i = x | X n ) = 1 


x~X n 


A (X n ,x)Z(n,x) 


T, w ~x n A (X n ,w)Z(n,w) ‘ 

It is shown in Pemantle (1988a, 1992) that for generic symmetric values of A and finite 
graphs G, the vector of normalized occupation measure, (Z(n,v)/n)) v£ y^Q'j, must converge 
to an element of a set of equilibrium points which is typically finite. Unfortunately, the case 
where A is identically 1 is not generic, but rather degenerate from the point of view of the 
previous works, and so this one most natural case is left unanalyzed. 


While the results of Pemantle (1992) do not extend to the case in this paper, it was 
conjectured there that in such cases the range of VRRW will be finite, and that in fact it 
will get “stuck” in a set of 3 points. In this paper we show that this behavior holds, in 


the sense of normalized occupation measure, at least with positive probability (Theorem 1.3 
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below). If one cares about the set of points visited infinitely often, rather than with positive 
density, the size of the set on which the walk gets stuck is 5 rather than 3. We obtain the 
following further results about the size of the range. 

Theorem 1.1 Let R = {k : X n = k for some ?r} be the (random) range of the process 
Xq,X\, .... Then P(|i?| = 5) > 0 and P(|i?| < oo) = 1. 

Theorem 1.2 Let R' = {k : X n = k infinitely often} be the essential range of the process 
X Q , Xx,.... Then P(|i?'| < 4) = 0. 

Remark: Simulations appear to show that P(|i?'| = 4) is nonzero, but these are evidently 
misleading. 

We conjecture but cannot prove that P(|i2'| = 5) = 1. It is easy to see that if R' = 
{k, k + 1, ..., k + j}, then Z(n, k)/n and Z(n, k + j)/n both converge to zero, or in other 
words, the occupation density goes to zero at the endpoints of the range. Quantitatively, 
we have 

Theorem 1.3 For any closed interval I C (0,1) and any integer k, there is with positive 
probability an a € I such that the following six events occur: 

( i ) R! = {k — 2, k — 1, k, k + 1, k + 2}; 

(ii) log Z(n, k + 2)/ log n —» a; 

(in) log Z(n, k — 2)/ log n —> 1 — a; 

(■ iv ) Z(n, k + l)/n —> a/2; 

(n) Z(n,k — 1 )/n —> (1 — a)/2; 

(vi) Z(n,k)/n —> 1/2. 

We conjecture but cannot prove that this is the universal behavior, i.e., that there is always 
such an a£ (0,1). 
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The power law behavior in parts (ii) and (in) of Theorem 1.3 rests on the analysis 
of a certain interacting urn process. This urn process is of a type studied in the doctoral 
dissertation of Athreya (1967), via embedding in a multiptype branching process. Since this 
is not generally available, and since our hypotheses and methods of proof are quite different, 
we include complete statements and proofs of the relevant results. The next section gives 
some background on urn processes and reinforced random walks. Proofs for the results on 
urns are given in Section 3, and proofs of Theorems BIO and |1.3| are given in Section 4. 
The final section completes the proofs of some lemmas and poses a few open questions. 


2 Background on urn processes and processes with reinforce¬ 
ment 

This section begins with a brief survey of previously studied reinforced random processes. 
By popular demand, we have included more of a review than is strictly necessary for the 
analysis of VRRW, that being the generalization of Theorem stated and proved in the 
next section. 

The simplest (and one of the oldest) process with reinforcement is known as Polya’s urn, 
after the 1927 paper of Eggenberger and Polya. In this model, there is an urn containing 
red and blue balls. At time 0 the urn contains r red balls and s blue balls. At each time 
k > 1, a ball is chosen uniformly from the contents of the urn, and is put back into the urn 
along with a extra balls of the same color. Thus if X n denotes the number of red balls at 
time n and Y n denotes the number of blue balls at time n, the dynamics are governed by 

y 

(X n+ i,Y n+ i) = (X n + a, Y n ) with probability ——; (2.1) 

■A-n i j-n 

Y n 

(X n+ i,Y n+1 ) = (X n , Y n + a) with probability —— 

■A-n i *n 

Eggenberger and Polya showed that the proportion of red balls, Z n := X n /(X n + Y n ), 
converges almost surely, and that the limit is random. The distribution of the limit is a 
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beta with parameters r/a and s/a (thus uniform over [0,1] when r = s = a = 1). This 
random limit behavior is possible because Z n is a martingale. In the sections to follow, we 
make use several times of this elementary principle: 

Proposition 2.1 If Z n = X n /(X n +Y n ), and if (X n+ i,Y n+ i) = (X n +\,Y n ) with probability 
Xji/(Xiyi V u ), and (Xy,, V,, -f- 1 ) otherwise, then E( Z,,_|_\ | Z,, ) — Z n . Also, Z,,_|_\ Z n | *7 
l/(X n + Y n ). □ 

In contrast to this is an variant suggested by B. Friedman (1949), where in addition 
to the a extra balls of the same color, one also adds b balls of the opposite color. This 
produces strikingly different behavior, even when b « a. Freedman (1965) showed that 
Z n —» 1/2 almost surely, with ( Z n — 1/2)/n -7 converging to a nontrivial distribution for 
7 > 0 depending on a and b. To explain the differing behavior, note that { Z n } is not a 
martingale, but rather 

E(^ n +i — Z n | Z n ) = n 1 (f(Z n ) + o{ 1)) (2-2) 

where / is a function vanishing only at 1/2. One could say that the drift, /, pushes { Z n } 
toward 1/2, which is an attracting point for the one-dimensional vector field given by /. 

When discussing processes with reinforcement, it is good to keep in mind the distinction 
Polya-like (/ = 0) versus Friedman-like (/ / 0 except at isolated points), which dictates 
the important aspects of the long-term behavior. A third category, singular , occurs when 
/ has zeros on the boundary, in which case convergence happens at a slower rate. 

The prototypical Friedman-like model is Robbins and Monro’s (1951) stochastic approx¬ 
imation scheme, which obeys the law 

Fj(Z n ~ i-i — Z n | Z n ) = n 1 F(Z n ) 

for a generic function F about which imprecise information can be obtained by sampling. 
The (unknown) zeros of F are then “found” by the {Z n j process. Since the 1950’s, stochas¬ 
tic approximation has been an active research area; the overview by Kushner and Yin (1997) 
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gives an idea of progress and techniques in stochastic approximation since then. The lit¬ 
erature on formal models of learning contains many Friedman-like processes, in which the 
transition probabilities of a finite state (non-Markov) chain are updated based on the some 
kind of objective function. The chain then “learns” to spend most of its time at states with 
large values of the objective function. The first round of this literature appeared in the late 
sixties, e.g., Iosifescu and Theodorescu (1969), and a second round emerged with the study 
of neural nets. The common theme is self-organization by a system whose basic parameters 
are extremely simple. 

Polya-like models have appeared frequently in theoretical statistics, due to the fact 
that bounded martingales are mathematically equivalent to sequences of posteriors, given 
increasing e-fields. For example, suppose an IID sequence of zeros and ones has an unknown 
mean p. with the prior on p being uniform on [0,1] (or more generally, any beta distribution). 
Then the sample sequences { Z n } of a Polya urn process can be interpreted as posterior 
means, where each red ball picked corresponds to observing a one and each blue ball picked 
corresponds to observing a zero. The so-called Bayes-Laplace estimate of the probability 
the sun will rise tomorrow and Greenwood and Yule’s (1920) model for industrial accidents 
are both based on this interpretation. In modern times, Blackwell and McQueen (1973) 
contruct Ferguson’s Dirichlet via an urn process, and Mauldin, Sudderth and Williams 
(1992) use a tree full of urns to construct a family of priors on distributions on [0,1] with 
nice properties. 

Polya-like models have also arisen in modeling of self-organization and random limits. 
Arthur (1986) and Arthur et al (1987) use both Polya- and Friedman-like urns to model the 
growth of industry and explain random clustering and market share patterns. Reinforced 
random walks were introduced by Coppersmith and Diaconis (1987) as another, somewhat 
simplified model of self-organized behavior. Although simplistic, urn models and reinforced 
random walks have been taken seriously in the modeling of physical phenomena; see for 
example Othmer and Stevens (1998), in which motion and aggregation of myxobacteria 
along slime trails are modeled by reinforced random walks and related stochastic cellular 
automata. 


5 



The VRRW studied by Pemantle (1988a , 1992) is a variant of their edge-reinforced 
random walk (ERRW). In ERRW, one keeps track of the number of times each edge has been 
crossed, the augmented occupation numbers being denoted {Z(n, {u,w}) : {v,w} € E(G)}, 
and one chooses the next edge from among the edges adjacent to the present vertex, with 
probabilities proportional to the augmented occupation of each edge: 


P(^n+l 


x\J~ ri) — ^-X^X n 


Z(n , {X n ,x}) 
E w ~x n z (nAX n ,w}) ' 


Reinforcing edges rather than vertices makes a dramatic difference in the behavior of the 
process, because edge-reinforcement is Polya-like and vertex-reinforcement is Friedman-like. 
A depiction of this difference via simulation may be found in Othmer and Stevens (1998). 
To see how to account for the difference theoretically, let v is a vertex in an acyclic graph, 
with incident edges e \,..., e*,. The successive edges chosen each time v is visited form a 
Polya urn process and it is not hard to see that these are independent as v ranges over all 
vertices. (The analogue of this fact on a graph with cycles is much harder to formulate and 
prove.) Coppersmith and Diaconis (1987) proved that the normalized occupation measure 
of ERRW on a finite graph converges to a random vector having a nonzero density with 
respect to Lebesgue measure on the simplex. When G = Z, Pemantle (1988b) shows that 
the process is a mixture of positive recurrent Markov chains, and in particular, that the 
normalized occupation measure converges to a limit that is everywhere positive. 


The question of the behavior of either VRRW or ERRW on a lattice of dimension two or 
greater is still open. Some progress on ERRW has been made by generalizing the model so 
that the k th crossing of each edge adds a*, to the occupation, where {a*,} is a pre-specihed 
sequence (a^ = 1 in standard ERRW). A general recurrence/transience dichotomy for this 
model was obtained by Davis (1990) in one dimension, while Sellke (1994) has results on 
the coordinate processes for this model in two dimensions. 


Our results for one-dimensional VRRW depend on an analysis of an urn model gen¬ 


eralizing both the Polya and the Friedman urn. Replace the dynamics (2.1) by the more 
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general dynamics: 


{X n + 1 ) Yn+ 1 ) — {X n + cl, Y n + b) 
(X n+ \, in+l) = (X n + C, P n + d) 


with probability 
with probability 




V n + Y ’ 




V n + Y n 


(2.3) 


There is no assumption that the number of balls be integral. When (“ jj.) is a multiple of 
the identity matrix, we recover Polya’s urn, and when a = d and b = c are all nonzero, we 
recover Friedman’s urn. In any case where (“ has an eigenvector (ui, V 2 ) with positive 
components, Freedman’s analysis can be carried through to show that X n /(X n + Y n ) con¬ 
verges to v\/{v\ +U 2 ). Perhaps the cleanest way to do this is via embedding in a branching 
process, as described in Athreya and Ney (1972, chapter V, sec. 9). Thus in particular this 
holds when be > 0. Two interesting cases are the singular cases, which can be reduced with¬ 


out loss of generality to the cases in the next two theorems. Theorem 2.2 was first proved 


by Athreya (1967) in a different form, while Theorem 2.2 is derivable from his results. 


Theorem 2.2 Suppose a > d = 1 and b = c = 0. Then X n /Y£ converges almost surely to 
a random limit in (0, 00 ). 


Theorem 2.3 Suppose a = d = l,b = 0 and c > 0. Then X n /(cY n ) — logY n converges to 
a random limit in (— 00 , 00 ). 


Remarks: (1) Theorem T3 is in a sense a finer result than Theorem 2.2 , since it deals with 
the second order correction: Y n is like n/ log n multiplied by a specific constant, with a 
random correction of lower order: X n ~ cY n (A + log Y n ), where A is random. (2) The class 


of urns in Theorem 2.3 is not needed for analysis of VRRW on Z, but is relevant to VRRW 
for a different reason. In the case c = 1, there is an isomorphism between the urn process 
and VRRW on the graph G with V{G) = {A, B}, having one edge between A and B and 
one loop connecting A to itself. Thus VRRW on G spends roughly time n/logn at B up 
to time n. 
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3 Urn model proofs 


This section is devoted to proving Lemma |3.5| , which generalizes Theorem 2.2 to allow 
random increments. Whereas Athreya (1967) proved version of this by embedding in a 
multitype branching process, we use martingale arguments (also considered by Athreya in 
some subcases). These turn out to be easier in the case of Theorem |2.3| than in the case of 
Lemma [b^ below. Consequently, we first give a relatively short proof of Theorem |2.3| and 
then state and prove Lemma |3.5| . Depending on your tastes, you may find the shorter proof 
or the more modular general proof easier to follow. Begin with the following easy lemma. 


Lemma 3.1 Let the nonnegative matrix (“ %) satisfy (a + c)(b + d) > 0 and define an urn 
process as in (\2.tf). Then min {X n ,Y n } —> oo almost surely. 


Proof: The proportion of red balls at time n is always at least Xq/{X o+YQ+n(a+b+c+d)). 
Since the sum of these quantities is infinite, the Borel-Cantelli Lemma tells us that a red 
ball is chosen infinitely often. Similarly, a blue ball is chosen infinitely often. After each 
color has been chosen k times, min{X ?l , Y n } is at least k min{a + c, b + d}. □ 

The following general fact about convergence of random sequences is also useful. 


Lemma 3.2 Let {Z n : n > 0} be a random sequence measurable with respect to the filtration 
{Xn}. Define 

A n = E(Z n _|_i — Z n \T n ) ; Q n = E((Z n+ i — Z n ) 2 | X n ). 

Then as n goes to infinity, Z n converges to a finite value almost surely on the event Y^ n A n < 
oo and Jfn Qn < oo. 

Proof: Let tm be the first time n that Y^j= o Qj > M. Let 

nf\TM 

Zi M) = z„ AT „ - Y, A i ■ 

3=0 
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Observe that {++} is a martingale with 

E((zgg - zf ) 2 I X n ) < Var(Z n+1 - Z n | T n )\ TM>n < Q n l-r M >n 

and so converges almost surely and in L 2 to a finite limit, Cm- On the event {]P n Qn < 
oo}, tm will be infinite for sufficiently large M, and the sequence {Z n } will converge to 


Proof of Theorem 1^3]: Let 

Z n = -^ 7 - - log Y n . 
cY n 

We wish to apply Lemma |3.2j to {Z n : n > 0}, so we must compute A n and Qn- It will turn 
out that A n = 0(1/Y n 2 ) and Q n = 0(n/Y,'£), so we are going to need a preliminary lower 
bound on the growth rate of Y n in order to see that these are almost surely summable. 


Lemma 3.3 For any e > 0, the function X n /Y^ +e is a supermartingale when X n and Y n 
are both at least c + 2. It follows that Y n is almost surely eventually greater than any power 
of n less than 1. 


Proof: To see that X n /Y^ +e is a supermartingale we compute the expected increment. 


E 


X 


71+1 


X n 


Y 


l+e 


n+1 


Y 


l+e 


■Fn 


Xn 1 Y n C _ Yn X n ((Y n + l) 1+e - Y^ +e ) 

X n + Y n Yn +e X n + Y n Y n 1+£ X n + Y n Y n 1+e (Y n + l)!+ e 


= —7TT7T+I (Xn + cYn - X n Y n - (Y n /(Y n + l)) 1+e ) . 

\^-n 1 i n )in 

This is nonpositive when min{X n , Y n } > c + 2, proving that X n /Y^ +e is a supermartingale 
under this condition. By Lemma |3.1| , both X n and Y n converge to infinity, so there is an 
almost surely finite N = N(e, w) such that min{X n , Y n } > c+2 for n > N, and consequently, 
{Xn/Y^ : n > N} is a super martingale. This implies that limsup n X n /Y^ +e < 00 , and 
hence for any 0 < e < 5, that lim sup n Xn + ^ /Y n = 0 , proving the lemma. □ 
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We continue with the proof of Theorem 2.3. We first compute the expected increment 
A n := E(Z n _|_i — Z n \J- n ) of Z n . 


An = 


An 


1 + K 


C Y n 


Xn + Y n cY n X n + Y n cY n X n + Y n \ ch n (l ri + l) 


AT 


- log 


Y n + 1 
Y n 


1 


A,-, 


X n + \cY n 
1 ( A 


+ 1 - 


A„ 


= 0 ( 


X n + Y n \ cY n (Y n + 1) 

1 , 


c(Y n + 1) 

+t 4 ) ) 


- Y n io g (i + i/y„)^ 


Y 2'" 


(3.1) 


Now compute an upper bound for the quadratic variation Q n := E((A n +i — Z n ) 2 | A n ) as 
follows. 


Qn — 


An 


1 


+ 


An 


1 


An 


1 


A n + An c 2 A n 2 ' An + A n V Y n cY n (Y n + 1) log(1 + An 


1 1 n . 1 , An An 

< — + -n +l0g 2 (l + — ) + 


c 2 A 2 A 2 


Yn An + An C 2 An 4 


< 


n 


CAn 3 


(3.2) 


for an appropriate constant C , using the fact that X n < (1 + c)n + Ao- We are now done: 
Lemma together with (0) and ( |3.2| ) show that A„ and Q n are almost surely summable, 
hence the conclusion of the theorem follows from Lemma □ 


Proof of Theorem 2.2: We now prove Lemmas 3.4 and |3.5|, which together imply as a 


special case a result in the spirit of Theorem 12.21. We use supermartingales similar to those 


in Lemma 3.3. 


Lemma 3.4 Let (A n ,Y n ) be a positive process converging coordinatewise to infinity. Fix 
any (3 > 1 and suppose there is an M = M(/3) < oo such that Y@/X n is a supermartingale 
once A n , A n > M. Then 

limsup ^° g -" < \ on {M < oo} . 
n log An P 
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Similarly, if X n /Y(/ is a supermartingale once X n ,Y n > M', then 

liminf > 1 on {M < 00 } . 

n logX„ (3 

PROOF: Let r m be the least n>m for which min{X n , Y n } < M. Then 

{YnATm/ X nATm ■ n > m} 

is a nonnegative supermartingale, so converges almost surely to a limit L(m). When r m = 
00 , it follows that L(m) is the almost sure limit of Y///X n . Thus when r m = 00 , Y n = 
[(( L(m ) + o(l))X„] 1 /^. If L(m ) > 0 this implies lirri sup log Y n j log X n = 1/(3, while if 
L(m) = 0, the lim sup may be strictly less than 1/(3. On the event {M < 00 } an m 
exists with r m = 00 , which finishes the proof of the first assertion. The proof of the second 
assertion is similar. □ 


Lemma 3.5 Let (X n ,Y n ) be a process generalizing the urn process in Theorem 2.k as fol¬ 
lows. For each n, with probability X n f ( X n +Y n ), there is a W >0 such that X n+ \ = W+X n 
and Y n+ \ = Y n ; with probability Y n /(X n + Y n ), we have X n+ i = X n and = Y n + 1. 
Suppose further that with probability 1, 


E(W\F n ,X n+1 >X n ) € [a, b] 


(3.3) 


and 

E(W 2 \F n ,X n+1 >X n ) <K (3.4) 

for some positive constant K and some 0 < a < b. Then for any 0 < (3 < a, there is an M 
such that whenever Y n > M, the function Y///X n is a supermartingale. Likewise, for any 
(3 > b there is an M' such that X n /Y// is a supermartingale whenever Y n > M'. 


Proof: By a Taylor expansion, there exist constants c\ and C2 such that for any w, and 
any sufficiently large x, 


1 1 w ur 

—I— —-y + c i ~3“ 

x + w x x- aP 
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Also. 


(y + i)^ < / + A/ 3 ' 1 + c 2 /3/ 


-2 


The expected increment A n := Y^ +l /X n+ 1 — YfX n , conditional on iF n , is given by 


X„ 


-E 


Y? 

n 


Yf 


X n + Y n \X n + W X n 


+ 


Y n 


X n + Y n 


-E 


'(Yn + lf Yf 


Xn 


Xn 


Plugging in the Taylor estimates above yields 
EA n < 


Y(_ EW+c ™ r 


Xn + Y„ XI y 

The assumptions on W imply that 


X„ 


+ 


Yn (3Yff C2_ 
Y n 


X n + Y n X , 


EA n < 


yd 

1 n 


(X n + Y n )X n 


C 2 CliP 

(3~ a + — + —— 

1 n. 


When (3 < a and X n and Y n are sufficiently large, then this is nonpositive. Choosing M 
large enough so that the constants ci and C 2 in the Taylor expansion are valid whenever 
X n . Y n > M, we have proved the first assertion of the lemma. 

The proof of the second assertion is similar. Choose c so that 

(y + ir^y-P-Py-P-Hi--) 

y 

whenever y > 1. The expected increment A n := X n+ \/Yf x — X n /Yf conditional on T n , 
is given by 


X, 


n W 

E— + 


Yn 


E 


X„ 


Xn 


Xn + Y n yf X n + Y n \ (Y n + If Yn 7 ' 


Thus 


EA n < 


X„ 


(Xn + Y n )Yg 
proving the lemma for M' = (3c/(b — (3). 


b-0 + y-) , 


□ 


Finally, we show Lemma [LJ and Lemma |3.5| together imply the first order of approxi¬ 
mation in Theorem namely that \ogX n /\ogY n — 1 ► a almost surely. The urn process in 
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Theorem satisfies the conditions of Lemma |3.5| with K = a? and [a, b\ = {a}. Thus for 


any 0 < (3 < a, we may plug the conclusion of Lemma L5 into Lemma |3.4| to see that 

log Y n 1 
lim sup --— < - . 

n log X n 


Similarly, for any f3 > a, we plug the conclusion of Lemma 3.5 into Lemma 3.4 to see that 

n . - f log^n ^ 1 
n log X n (3 

Since j3 may be chosen arbitrarily close to a, we see that \ogY n /\ogX n —* 1/a as n —> oo. 

□ 


4 Proof of Theorem 11.31 


In the next section we will prove the following lemma: 


Lemma 4.1 There is an e > 0 such that for all integers m > 0, 

P (m + 3 £ R\m £ R) < 1 — e. 


The first statement of Theorem 0 follows from Theorem |L^. The second statement follows 
directly from Lemma 4.1: by induction, P(3n £ R) < (1 —e) n , which goes to zero as n oo. 
Hence we concentrate on the proof of Theorem |l.3|. Begin with a lemma. 


Lemma 4.2 Let J = [a, b] be an interval of integers containing zero. Let P denote the law 
of VRRW on Z as before, and let P j denote the law of a VRRW on the interval J, both 
started from 0. Then the following four conditions are equivalent. 

(*) P {R CJ)> 0; 

(ii) P (R! C J) > 0; 

(in) P (£n 1 x n = a Z (n, a + l)" 1 + 1 x n =bZ (n, b - 1) _1 < oo) > 0. 

(iv) P j (J2 n 1 x n = a Z(n, a + 1) _1 + Jfn lx n =bZ(n, b - 1) _1 < oo) > 0. 
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Proof: We define a coupling, i.e., a measure Q on pairs of paths ({X n : n > 0}, {X' n : n > 
0}) such that the first coordinate of Q has law P and the second has law Pj. To do so, 
choose {X n } according to P and let r be the first time n that X n e {a — l,b + 1}. Let 
X' n = X n for n < t, let X' T = a + 1 if X T = a — 1, let X' T = b — 1 if X T = b + 1, and let 
X' n be chosen from the transition probabilities for Pj independently of {X n : n > 0} when 
n > t. 


Observe that 


Q(t = n + l\r > n) = l X ' n = a Z(n,a + l) 1 + l x > n = a Z{n, a + 1) 

Thus by Borel-Cantelli, Q(t = oo) > 0 if and only if condition ( iv ) is satisfied. The 
event {r = oo} is the same as the event {i? C J}, proving the equivalence of (i) and (iv). 
Similarly, from the equation 

Q(t = n + 11 r > n) = 1 x n = a Z(n, a + 1) _1 + l Xn = a Z(n, a + 1) _1 


one sees that (i) and (in) are equivalent. The implication (i) =>■ ( ii ) is clear. Finally, to see 
that (ii) implies (in), assume (ii). Thus with positive probability, Z(n,a — 1) + Z(n, b+ 1) 
is bounded as n —* oo. By Borel-Cantelli, this means that 


P 


P(X n € {a - 1, b + 1} | T n ) < oo 


> 0 . 


This sum is an upper bound for the sum in (in), hence the sum in (in) is finite with positive 
probability. □ 


Corollary 4.3 Suppose that Z(n,a+ 1) and Z(n,b— 1) areQ(n), i.e., liminf Z(n, a+l)/n > 
0 and liminf Z(n, b — l)/n > 0. Then P (R C J) > 0 if and only if 



Z(n, a) + Z(n, b) 


< oo 


> 0 . 
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Proof: Let a m be the first n for which Z(n, a+ 1) = m and let p m be the first n for which 
Z(n,b — 1) = m. Summing by parts gives 

Y 1 x k = a Z(k, a + l)' 1 + Y 1 x k =bZ(k, b - 1) _1 
k k 

Z(o~ n -f- 1, cl) Z(a n , cl) Z(p n -f i, b) Z ( p n , b) 

n n 

_ Z(a n ,a) 1 Z(p n , b) 1 

n 2 — n n 2 — n 

Since Z(n,r) is increasing in n for all r and we have assumed cr n , p n = 0(n), this proves 
the corollary. □ 



Proof of Theorem 1.3: There are four steps to the proof. The first is to reduce to a 
VRRW on the five points —2, —1,0,1 and 2. The second is to show that this VRRW can, 
with positive probability, have 2Z(n,l)/n remain in the interval /, while simultaneously 
Z(n, 2) and Z(n,— 2) remain less than n 1 ^ 6 for a prescribed e = e(I) > 0. The third step 
is to show that when these two things happen, then actually 2 Z(n, 1 )/n converges to some 
a € I. The fourth step is to see that whenever 2 Z(n, 1 )/n converges, then Z(n, 2) almost 
surely obeys the power law 

lim l0gZ(n ' 2) = lim 2Z(n ’ 1) . 

n —xx) }og Tl n —^°° 77- 


Step 1. This step is essentially done. If we show that for J = {k — 2,k — l,k,k + l,k + 2}, 
the Pj probability of properties (ii) - ( vi ) holding simultaneously is positive, then the 
conclusion of the theorem follows from (ii) - ( v) and Corollary The argument is the 
same for every k, so from now on we assume without loss of generality that k = 0, and set 


about proving Theorem 1.3 for Pj in place of P, where J = {—2, —1, 0,1, 2}. 


Step 2. For the remainder of the argument, fix an interval I = [c, d] C (0,1) and a 
positive e < min{c, 1 — d, d — c}/10. Let /3 = (1 — e) . Also fix an integer Nq and 
define stopping times depending on Nq as follows. Let be the least n > Nq such that 
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2 Z(n,l)/n £ I. Let T2 be the least n > Nq such that Z(n, 2) > n l ~ e and let 73 be the 
least n > Nq such that Z(n, —2) > n 1_e . Let r = n A To A 73 . Let {z* : —2 < i < 2} be a 
quintuple of integers. Our goal in this step is to identify an Nq and a quintuple z* such that 

Pj(t = 00 | Z(Nq, i) = Zi : —2 < * < 2) > 0 ; (4-1) 

in fact we will show it is near 1 . We assume fj for the moment, and continue with steps 3 
and 4. 


Step 3. Let K n be the time of the n th return to the state 0 and define 

y = Z(Kn,l) 

Z {uni 1 ) + Z (n n , — 1 ) 

We will see below that when n n < r 2 A 73 , 

\Ej(V n+ 1 -V n \T Kn )\ < Ck - 1 " 6 ; (4.2) 

E./((14 + 1 — V n ) 2 | T Kn ) < C'k~ 2 . (4.3) 


Plugging these two bounds into Lemma 3.2 shows that whenever r = 00 , the sequence V n 
must converge, to a value necessarily in I. This gives us parts (iv) and (v) of the theorem, 
with part (vi) already following from step 2. 


Step 4. We claim that for fixed r and s, whenever 

r < lim inf V n < lim sup V n < s (4-4) 


and r = 00 , then 


1 ■ t log Z (n, 2) logZ(n,2) 1 

— < lim ml —--< lim sup —-- < — 

s log n log n r 


To prove the claim, define the return times {a n : n > 0} to state 1 by letting a n = min{n > 
a n -i : X n = 1}, and a_i is set equal to iVo — 1, for some TVq Let 


U n = Z(a n , 0); 
U' n = Z(a n ,2); 
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For any S > 0, we show that the conditions of Lemma 3.5 are satisfied with (X n ,Y n ) = 
(U n . U' n ) and [a, b] = [r — 6, s + <5]. Indeed, between times a n and a n +i, VRRW will either 
visit state 2 once or will visit state 0 some number of times W > 1. The probabilities of these 
disjoint cases are respectively U n /(U n + U' n ) and U' n /(U n + U' n ). Let N\ be the least N > Nq 
such that r — 5 < inf n >jv V n < sup n>N V n < s + <5; when ([4.4D holds, Nj will be finite. We 
need to show that when n > N\, then ( |3.3| ) and (3.4) hold, with [a, b] = [r — 5, s + h]. Since 
P j(W > k) is equal to the probability that on the first k — 1 visits to state 0 after time a n 
the VRRW moves to the left, the assumption that n > N\ implies that 

(1 - s - S) k < P j(W >k)<{l-r + 5 ) k , 

which gives l/(s + 6) < E./VF < l/(r — 6) and E jW 2 < K for some constant K = K(r,5). 
The conclusion of Lemma 3.5 is that y^/( s+2< h jx n and X n /Y r \^ r 2< ^ are supermartingales 
for n > M, where M will be finite when Q4.4| ) holds. We then apply Lemma 3.4 together 
with the fact that a n = 0(n ) on {t = oo} to see that 

1 


s + 25 


. f logZ(n, 2 ) logZ(n, 2 ) 1 

< lim ml —7 - < lim sup —r- < 


log n 


logn 


r — 25 


on (4.4) when t = oo. Sending 5 to 0 proves the claim. 


Applying the claim simultaneously to all intervals (r, s ) with rational endpoints, we see 
that conclusion (ii) of Theorem |1.3| holds with probability 1 whenever r = oo. An identical 
argument establishes conclusion (in). Since we have shown that Pj(r = oo) may be made 
arbitrarily close to 1 by suitable choice of {zi : — 2 < i < 2 }, we are done with all four steps, 
modulo the verification of and of ([4 . 2|) and (|4.3|). 


Cleanup step. First we prove (|4.2[) and (|4.3| ). Let A n = V n+ \ — V n . We estimate A n in 
three pieces. Let A be twice the least integer greater than 2/e. Write A n = R n + S n + T n 
where 

Z ( K n + 2 , 1 ) 


Rn = 


S n = 


T n = 


Z (n n + 2 , 1 ) + Z ( K n + 2 , — 1 ) 

z(nn+l A (ftn + -4)) 1) 

Z (/v n +i A (ec n + A), 1) + Z(k u + i A (n n + A), — 1) 
R n S n . 


— V n — Rn 
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By Proposition 2.1 , E j(R n \ J ~ K .„) = 0 and < n n 2 . By the same token, S 2 < A 2 n n 2 and 
we easily see that 


A 


2 N, 


l—e 


A 


| J~ K „) < P j( K n+l > Kn + 2 | •7 r /trl ) < o; . r i_ e i_|_ e 

when K n < T 2 A T 3 (as n/2 — n 1-e < n n < n and Z(n n , 0) = /c n ). Finally, since T n < 1, we 
have 

Ej(|T n |* | JF Kri ) < Pj(k „ +1 > K n + A | R Kn ) 

for i = 1,2. The RHS is just the probability of at least A/2 successive moves from state 1 
to state 2 or state —1 to state —2. This probability is at most the maximum of 

Z 2 ) + * 


n 

i<A/2 


Z(unt 2) + Z (fv n , 0) + i 


and the same expression with 2 replaced by — 2 . Since < k~ 2 by choice of A, and 

since the terms in the product are at most a constant multiple of k~ £ by the assumption 
that n n < 72 A 73 , the RHS is bounded by a constant multiple of k~ 2 . Having bounded 
the conditional expectations of R^, S 2 and T 2 by multiples of k~ 2 and the magnitudes 
of the conditional expectations of R n ,S n and T n by constant multiples of we have 


established (4.2) and (4.3). 


To establish (|P| ) , we will show that all of the three probabilities Pj(t 2 < t\ < 00 | Tn^))i 
P,/(t 3 < T\ < 00 | Rn 0 ) and Pj(ri < 72 AT 3 | J~n 0 ) are simultaneously small when the values 
Z(No,i) = Zi are chosen appropriately. As in step 4, we define the return times a n to 
state 1 by a_ 1 = Nq — 1 and a n+ \ = minjfc > a n : = 1}. Again set U n = Z(a n , 0) 

and U' n = Z(a n , 2). As in step 4, the process (U n ,U!^) evolves as the urns in Theorem |2.2| , 
where again we let W = U n+ \ — U n . Assume that a n < t\ A 72 . A lower bound for the 
probability that W > K is the probability that from state 1 the VRRW visits 0 and then 

visits states —1 and 0 I\ times in alternation. Thus 

K 


Ej(W\R an ,W>0) > l + J2VAW>i-l\R an ,W>0) 

Z(cx n , — 1 ) +j- 1 


i=2 
K i -1 


> i+zn 


^2 j=i — 1 ) + Z(a n , 1 ) + j — 1 
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There is a 4>{K) such that when Nq > (f>(K) and a n < t\/\t 2 then each factor in the product 
is at least 1 — d — 2e. Thus for sufficiently large K and a n < r± A t 2 , the RHS is at least 
l/(d + 3e). It is trivial to see that E./(IF 2 | T n ) is bounded. Thus setting a = 1 /(d + 3e) 
and (3 = l/(d + 4e), we apply Lemma |3^ to see that {U' n f /U n is a supermartingale when 
a n < Ti A t 2 . More formally, let p be the least n for which a n > t\ A t 2 . Setting Y n = U' nfyp 
and X n = Un^p, the process /X n is a super martingale. Since d + 4e < 1 — e, we see that 
by definition of t\ that Y^ jX n > 1 if 12 < t\ < 00 , where n is the least j for which otj > t\. 
Therefore, by the supermartingale optional stopping theorem we arrive at 

l/(eH-4e) 

p j(r 2 < n < 00 I J~ No ) < — -. 

^0 


An entirely analogous argument with the states —2 and —1 in place of 2 and 1 and c in 
place of 1 — d yields the analogous bound 


l/(l-c-4e) 

Pj(t 3 < n < 00 1 x No ) < — - 

Zo 


Finally, we need to see how to make Pj(ti < t 2 A 73 | Xn 0 ) small. Let p be the least n 
for which n n > t 2 A 13 . Then by (0), 


|E W p -Wo\< J2 CK n 1_e < Ci(e)iV 0 - £ . 

m>K n 

Similarly, (|4.3|) gives 

Var (W p - Wo) < J2 C '^ < C[N^ . 

m>K n 

On the other hand, if T\ < t 2 A r 3 then 


| W p - VFol > min{Wo -c,d-W 0 }. 
Chebyshev’s inequality applied to W p — TLo then shows that 

P j(n < t 2 A r 3 ) < 


C[N^ 


(min{IT 0 -c,d-W 0 }~ A ^) 2 ' 

When A'q is sufficiently large, and 2^i/A ^ 0 is sufficiently close to (c + d)/ 2, this is at most 
QNq 1 . Thus we have shown how to pick z - 2 ,..., z 2 so that P(r < 00 | JOv 0 ) can be made 
arbitrarily small, which finishes the proof of (4.1) and of Theorem |1.3|. □ 
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5 Remaining proofs and open questions 


The proof of Lemma 4.1 is quite similar to the proof of Theorem |1.3|. We give an outline 


for the argument, leaving out details that are the same as in the proof of Theorem 1.3. 


Sketch of PROOF of Lemma 4.1: The first reduction is to analyze VRRW on (—oo, m+ 2], 
Let T m be the first time m is reached. The hardest part, because it requires a simultaneous 
induction on two different stopping times, similar to ( 0 ), is the following: 


Claim 1: There is a constant S > 0 such that for all m, the probability is at 
least 5 that inequalities ( i ) and ( ii ) hold for every n > r m : 

(i) Z(n,m) > Z(n,m + 2) 2 ; 

(ii) Z(n, m — 1) > 2Z(n, m + 1). 

The other essential ingredient is 


Claim 2: Z(n,m+ 1) < (l/4)Z(n, m + 2) 2 ].^ finitely often almost surely, where 
Am is the event that (i) and (ii) of the previous claim are true for all n € [r m , M). 

Assume these two claims and let cr^ be the time of the k th visit to site m + 2. From the 
first claim, the decreasing limit A a0 has probability at least 5. Since Z(ak,m + 2) = k + 1, 
it follows from the second claim that on Aqo, 

OO 

Z(a k ,m + l)^ 1 < oo. 

k =1 

Fix M,e > 0 such that with probability at least e, J2kLi ^( a ki m + l) -1 < Af. As in 
Corollary [O] it then follows for VRRW on Z that P(m + 3 R\m £ R) > 0 and in fact 
that a lower bound is 5 := eexp(—2 M). 

To prove the first claim, stop the walk the first time either condition (i) or (ii) is violated. 
Consider first the process {(U n ,V n )} := {(Z(p n ,m — 1 ), Z(p n ,m + 1))}, where p n are the 
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successive hitting times of site m. At each step, precisely one of the coordinates is updated, 
with Polya-like probabilities, so by Proposition ^d], the expected increment of U n /(U n + V n ) 
is given by the contributions from increments of magnitude greater than 1: 

E Un+l _ Un_ = E (Un +1 ~U n ~ 1) + ~ (K+l ~ K ~ 1) + 

Un+1 T Vn+1 U n -p V n U n+1 T kn-|_i 

The term (U n+ \ — U n — 1) + is nonnegative and the term 


E 


-(V n+l -V n -l)+ 

U n +1 + K+1 


is of order 


1 Z(p n ,m + 2) , 1/2 

U n + V n Z(p n , m) 


by condition (ii). This expresses U n /(U n + V n ) as a martingale plus a drift term whose 
negative part is summable. 


Consider next the process {(U' n: V^)} := {(Z(p n ,m), Z(p n ,m + 2))}, where p n are now 
the successive hitting times of m + 1. Again the updates are in a single coordinate chosen 
with Polya-like probabilities, with the increment in being 1 and the increment in U' n 
having conditional mean at least 3. Using Lemma [L5|, just as in Step 4 of the proof of 


Theorem 1.3, we see that (Vh) 2 /U' n is a supermartingale 


The optional stopping theorem now shows that from an appropriate initial position, the 
probability of stopping due to a violation of (i) or (ii) is arbitrarily low. The initial position 
(or one at least as good) can be attained with a probability bounded away from zero (unless 
m is visited only finitely often, which is even better!) so the claim is proved. 


Finally, to prove the second claim, let X n = Z(p n+ i,m + 1) — Z(p n ,m + 1) where now 
p n are the successive hitting times of site m + 2. On the event A Pn+1 , the probability of a 
transition to m + 2 from m+ 1 between times p n and p n +i is bounded above by 1 /(n + 1) 
(use condition (i) and Z(-,m + 2) = n). Thus X n stochastically dominates a geometric of 
mean n, and it is easy to verify that 




k=\ 


n 


4 


finitely often, a.s., 
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which proves the second claim and hence the lemma. 


□ 


Proof of Theorem |1.2| : It is straightforward that P ( | /?/1 = 2) = P (| /? / 1 = 3) = 0, so 
we suppose that R! = J := {—2, —1,0,1} and show that this leads to a contradiction. To 
simplify notation, let A n , B n , C n and D n denote Z(n, x ) for x = —2, —1,0 and 1 respectively. 
The assumption R' = J yields that there exist N such that X n e J as soon as n > N. 
Throughout the rest of the proof, we assume that n > N. 

Let n m be the time of the m th return to —1 (clearly, K m ’s are < oo on the event R' = J). 
Between the times n m and n m +i, the random walk goes from —1 either to —2 and returns 
to —1, or to 0 and (possibly) bounces between 0 and 1 before going back to —1. Therefore, 
W m := A Km /(A Krn + C Km ) is a non-negative supermartingale which converges a.s. to some 
random variable W. Consider two cases: 


• Wiui) = w > 0; 

• W(u) = 0. 


In the first case, there exists N± > Nq such that W m > iu/2 for n > N\. As in Lemma 4.2 , 
the probability never to jump from —2 to —3 is positive only whenever 

A n +1 A r 


£ 


Bn 


< OO 


which is equivalent to the following sum being finite 




n>Ni B n 


Bn. — 


n —1 


Bn- 


n —1 


(5.5) 


(here we used the obvious inequality B n < A n + C n ). However, the sum in the RHS of (5.5) 
is a tail of the harmonic series and therefore diverges. 

Before we proceed to the second case we observe that by the same arguments we can 
restrict the problem to the case when both A n /{A n + C n ) and D n /(D n + B n ) go to zero. 
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As a result, A n /C n —> 0 and D n /B n —> 0 as well. Taking into account that B n < C n + A n 
and C n < B n + D n we conclude that 


B n 

C n 


2 Bn 


n 


1 , 


2 Cn 


n 


1 . 


(5.6) 


Let T m be the time of the m th visit to —1 or 0 skipping at least one step, 

Tm ■■= inf{n > r m _i + 1 : X n € {-1,0}}, r 0 := N 0 . 

Dehne 

B t +C t 

U m = m 2 m , V m = A Tm + D Tm . 

If X Tm = — 1 (X Tm = 0 resp.), then between the times r m and r m+ i VRRW will either (1) 
go to the left (right resp.) and back, or (2) go to the right (left resp.) and back, or (3) go 
twice to the right (left resp.) and make one step back. Consequently, either U m +i = U m + 1 

and V m+ i = V m (when (2) takes place) or U m+ \ < U m +1 and V m+ \ = V m +1 (when (1) or 

(3) takes place). We claim that the probability of the latter event, denoted by F, is greater 
than V m /(Um + L m ) when m is large enough. This, in turn, will imply that the process 
{U,V) can be coupled with some general urn model process described by ([E^) such that 
U m < X' m and > T ' m . 

To prove that P (F) > V rn /(U m + V m ) we consider the quantity A n — B n + C n — D n which 
is “almost” invariant for n > N. Namely, there exists a (possibly negative) constant K , 
depending on the history of VRRW before time N only, such that A n — B n + C n — D n = K 
whenever X n = — 1 and A n — B n + C n — D n = K + 1 whenever X n = 0. If we denote 
t m := B Tm + D Trn , then A Tm + C Trn equals t m + K or t m + K + 1 when VRRW is at —1 or 
at 0 respectively. In the second case 

pm = D f _ D\ A = V-AD/(t + K + 1) _ A(K + 1) 

1 ’ t + v t ) t +1< + 1 t t{t + K+ 1) 

(we omit the indices for simplicty). Taking into account that 2 U + V = 2t + K + 1, V > A 
and AD < V 2 /4, we obtain 

p(F) V > V~V 2 /(4t + 4K + 4) A(K+ 1) _ V 

{ ’ U + V - t t(t + K + 1) t + (V + K + 1)/2 
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> 


V{V + K - 1) 


V 2 


V(\K | + 1) 


t(2t + V + K + 1) i(4t + 4 K + 4) t(t + AT + 1) 

As m —> oo (and, therefore, n —» oo) we have —> oo and = o(t m ), whence 


is non-negative for m > M where M is some constant. The case X Tm = — 1 can be analyzed 
in the similar way. 


We have shown that for large m the process (U m , V m ) can be coupled with the process 
{X' m , Yjn) obeying the law ( |2 . 3|) with a = e? = 1, 6 = 0 and c = 1 such that U m < X' m and 
V m > Y-m- By Theorem |2.3| , there exists 

yr - l °s(K) e (-°°, oo) 

1 m 

which, in turn, implies the existence of a random variable £ G (0, oo) such that 


X'm < K log(2<X) for all m > M. 


Clearly, X' m /Y^ oo, so 

Vt 

y/ y 

m - log(2CX^) 

for all m larger than some Mi > M. Since £/ m < and the function f(x) = 

x/log(2 (x) is increasing for large x, V m > U m / log(2££/^J. Furthermore, r m —» oo and 
T m +i < T m + 3, so we asymptotically have 


by (|Q1) . 


A n + D n > 


{B n + C n )/2 ^ n 

log(C (B n + C n )) _ 2 log(n) 


The event that VRRW does not jump off J can occur only when the sum 


•sy A n+ \ A n ^ D n+ i D r 


B n 


C„ 
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is finite. Summing by parts as in Corollary 4.3, we obtain that this is equivalent to the 
finiteness of the sum 


A n + D n ^ (Bn + C n )/ 2 1 1 

„2 ^ COUSt + 2^ ^ V? tt 2 - COnSi + 2^ ■ 




log(C(B„ + C n ))n 2 ' ^2nlog(Cn) 

which diverges. Therefore, P (R' = J) = 0, completing the proof. □ 

We end with some questions. The strongest conjecture about VRRW on Z is the one 


stated after Theorem 1.3, to the effect that the behavior described in Theorem 1.3 happens 
with probability 1. Some smaller steps toward this would be to prove that the set of sites 
visited with positive density must be connected and to prove that a can never be 0 or 1. This 
would, for example, rule out that sites —1 and 0 are visited with density 1/2, and by time n 
the numbers of visits to the sites 1,2,... are asymptotically to/ log n, to/( log n log log to), .... 
Another graph on which VRRW may have interesting behavior is Z 2 . Ferrari and Meilijson 
(personal communication, 1996) also have some results about VRRW on a tree. 


A further question is that of stochastically comparing VRRW with different histories. 
For example, we originally thought we could prove a version of Lemma |4.1| in which it was 
shown that P(m + 3 £ R \ m E R) < P(3 G R), by showing that the extra weight to the left 
of m the first time m is reached can only help the range stay bounded above by m + 2. We 
were unable to do this, by coupling or martingale arguments, but believe that some such 
comparison must hold. The easiest to state are false. 
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